There are fundamentally two categories of musical synthesizers: (a) samplers (or “sampling synthesizers”), in which stored digitized recordings (or samples) of actual instruments are reproduced when notes are played on a keyboard connected to the sampler, and (b) synthesizers, in which sounds are created at the time they are played based on analog or digital electronic circuitry which creates the sound without reliance upon previously recorded actual instruments. These instruments today are predominantly polyphonic, meaning they can play more than one note at a time. While the nature of the invention is immediately more applicable to samplers, it will function in connection with synthesizers as well. For simplicity the discussion herein will focus primarily on sampling applications.
Current electronic musical instruments are predominantly sample-players, which means they play specially processed digital recordings of sounds in response to some sort of control input, typically a musical keyboard or a sequencer. In simple terms a sequencer is like a digital version of a player piano giving instructions to the sample player (or other electronic instrument) on which notes to play and how to play them. For the purpose of the instant invention, it doesn't matter whether a “real time” keyboard or other musical controller or a sequencer is used to play notes on the synthesizer. There are synthesizers in which waveforms are generated and/or manipulated to create sounds without any reference to actual recorded sounds (such as additive waveform synthesizers, fm-modulating synthesizers, and wave table lookup synthesizers, among others); these were the original types of synthesizers. Later, as digital audio technology developed and became affordable, samplers or sampling synthesizers became popular; for samplers, actual recordings of sounds are specially processed into files that are stored on digital media for later playback that emulates the original recorded acoustic instruments (or other sound sources).
Sampled sounds are sold in collections, or libraries, and the individual sounds in sample libraries may be created from recordings of one or several instruments. With ensemble instruments such as bands and orchestras, it is common for a group of similar instruments to be recorded together; this “multi-instrument” sound is saved as a single sample. Thus, a prior-art sample of the first violin section of a symphony orchestra may consist of a recording of sixteen violins playing the same note, and these same sixteen violins would then play another note, and the collection of such notes would be packaged and identified, for example, as the “XYZ first violin sample library.”
Depending upon the nature of the technology used in a prior art sampler, there may be a separate source recording (initial sample) in its library for each note the sampler is capable of reproducing, or a single note sample may be electronically interpolated to higher and lower pitches corresponding to various notes. The first option yields optimum sound quality, at maximum cost and complexity, to create the library and reproduce it in the sampler, whereas the second option yields lesser sound quality at a reduced cost and complexity.
When samples are initially recorded, there may be one or many instruments actually playing the sound (and each may be playing one or more notes). Typically with orchestral or large band sounds, entire sections of instruments play each sampled note, with all instruments in a given section concurrently playing a single note. Thus, in the prior art a sample of an orchestra section of eight cellos would be a single recording of eight cello players playing the same note. When this sample of one note is played back on a sampler, all eight instruments are heard playing the same note. Similarly, a sample of an orchestra section of sixteen violins would be made by recording sixteen individual violin players all concurrently playing the same note, and when this sample is played back the sound of all sixteen violins would be heard playing that note concurrently.
When prior art samples of sixteen violins are played back in a sampler, if the person playing presses one key on the keyboard (or otherwise causes one note to be played), the sound that comes out of the sampler is the sound of all sixteen violins playing that note. So far, this may be very close to what would be heard in an actual symphony hall where, if the conductor (or musical score) instructs the first violins to play that same note, all sixteen will play that note.
However, if the person playing the sampler with this prior art violin sample presses two notes on the keyboard (or otherwise causes two notes to be played), the sound that comes out of the sampler is the sound of all sixteen violins playing each of the two notes; i.e., one hears 32 violins playing; this is what is called “additive polyphony.” Additive polyphony is not what would be heard with an actual symphony because in that case (with our example) there are only 16 violinists present, not 32. In fact, when a conductor (or musical score) instructs such a first violin section to play two notes, half of the players (eight of them) will play the one note and the other half (the remaining eight) will play the second note.
If there are three notes to be played at once, the available players are split up into three groups, each group playing one of those notes. With sixteen players and three notes, obviously the division is not equal, so it would typically be done with one note assigned to 6 players, and each remaining note assigned to 5 players each. This is what is called “subtractive polyphony.”
When the number of players available cannot be evenly distributed among the number of notes being played, a choice must be made as to where the “extra” player or players are assigned. This choice can be considered to be top weighted if the extra player(s) play the highest note(s) or bottom weighted if the extra player(s) play the lowest notes(s). With live acoustic performers, the allocation decisions affecting which notes are given to available players is done through a process known as “divisi,” and the instructions for such divisi are created by any of several parties involved with the music creation. Any combination of the composer, a musical arranger, the conductor and the “first chair” player of the particular section of instruments typically decide who plays which notes; divisi is not an exact science or protocol in music, but it is a well-established and essential principle guiding live performances wherein more than one player of a particular type of instrument are playing at once.
As noted above, the prior art, when multiple notes are concurrently played on a sampler, multiple instances of the sampled recording are sounded. Thus, if one has a cello sample in the library made from eight cellos, and two notes are played together on the sampler, the sampler would play the sound of sixteen cellos playing, eight instruments per note. If one plays a triad (i.e., three notes concurrently) on the sampler, the sampler would play the sound of twenty-four cellos (i.e., three times the eight cellos per sample). Although this is what is available in professional studios, it results in an unrealistic sound quality which does not reflect how an actual orchestra would sound. By way of example, with a real orchestra, the power (or volume) of a cello section stays relatively constant whether the cello players play one or several notes simultaneously (e.g., the power is about the same whether eight cellists of an eight cello orchestra section all play the same note or if five are playing one note while three are playing a different note). With a prior art sampler, the power is multiplied approximately by the number of notes played. By way of another example, as more and more notes are played simultaneously with a sampler, the density of the harmonics sounded tends to create an organ-like effect rather than preserve the clarity and concise sound definition afforded by a reasonable and fixed number of instruments playing at once. (Note that there may be valid reasons to use additive polyphony, but optimum orchestral sound is not obtained using additive polyphony exclusively.)
The method by which prior art samples are implemented does not include any provision for automatic allocation of individual notes among a fixed number of players. Most conventional sample libraries have multiple players “built in” to a given sound sample and so the “additive polyphony” employed in typical samplers cause more instruments to be heard the more notes that are played at the same time. This causes the sound power to multiply with each note played (three notes played using a sixteen-violin sample will sound like the first violin section has suddenly grown to 48 players). For this reason, anyone who has tried to attain realistic or even usable “orchestral balances” using prior art samplers and sample libraries has had to constantly “ride gain” or adjust the volume level of the performance to compensate for the power build up with greater numbers of notes; such “gain riding” may alternately be done by skilful playing on a velocity-sensitive keyboard, but this can be an exhausting effort. In a “real” orchestra or other ensemble, such sound power (volume) build-up does not occur because no matter how many notes are played, there are only a fixed number of musicians and instruments on stage performing.
The realism of sampled sound also depends upon correct conveyance of the harmonic structure. Each instrument as played by a given musician produces its own unique timbre (harmonic structure) and these various harmonics together create the texture of the sound that is heard. With a fixed number of instruments constantly reallocated to whatever number of notes are being played (the “live” situation), these unique timbres are all present, but only one per instrument, and so the combined harmonic structure has a distinct and discernable quality to a trained ear. However, when this full set of instruments also play the next note and the next and so forth all at one time (the prior art sampler situation), the harmonic structures of these multiple sets of instruments playing various notes overlay one another, and the unique timbres are no longer discernable. The resulting sound may be described as “dense,” “organ-like,” or “muddy,” and no amount of volume control adjustment can remedy this unrealistic harmonic structure.
What is needed to improve the realism of sampled or synthesized musical performances is a way to allocate the notes played to individual instruments or to small groups of instruments, changing the allocations in accordance with the number of notes being sounded at any given time. That is the nature of the methods presented herein.